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Abstract 

Automated validation of flight-critical embedded systems 
is being done at the National Aeronautics and Space Admin- 
istration Ames Research Center Dryden Flight Research Fa- 
cility. The automated testing techniques are being used to 
perform closed-loop validation of man-rated flight control 
systems. This paper discusses the principal design features 
and operational experiences of the X-29 forward-swept- 
wing aircraft and F-18 high alpha research vehicle (HARV) 
automated test systems. Operationally applying automated 
testing techniques has accentuated flight control system fea- 
tures that either help or hinder the application of these tech- 
niques. The paper also discusses flight control system fea- 
tures which foster the use of automated testing techniques. 

Introduction 

Ames Research Center Dryden Flight Research Facil- 
ity (Ames-Dryden) is researching the application of auto- 
mated testing techniques for the verification and validation 
of man-rated flight control systems (FCSs). Automated test- 
ing techniques were applied to the X-29 forward-swept- 
wing aircraft (Fig. 1) and the F-18 high alpha research vehi- 
cle (HARV) (Fig. 2) to reduce the time required to reliably 
validate the control system. 

Automated testing techniques are being developed be- 
cause of the increasing cost of flight qualifying embedded 
systems. Relaxed static stability, supermaneuverability, and 
the optimization of handling characteristics have resulted 
in complex FCSs. Complexities will increase as manned 
hypersonic vehicles require vehicle management systems 
using distributed processing techniques. These techniques 
will integrate vehicle controls, propulsion, thermal manage- 
ment, hydraulic management, electrical load management, 
and mission planning. The number of test cases necessary 
to prove system dependability will increase exponentially. 
For example, the triplex redundancy management logic for 
the X-29 aircraft has approximately 90 inputs. If test cases 
of normal, null, and extreme failures are run, then the num- 


ber of test cases necessary to completely validate the redun- 
dancy management logic would be 3 90 . Given an average of 
15 min to configure and run a failure modes and effect test, 
the test cases would take approximately 2 .49 x 10 38 years 
to finish! Clearly, no system can be completely validated. 
New techniques must be developed to run more test cases 
in the same time. As the amount of test data generated in- 
creases, new methodologies for extracting information must 
also be developed. Embedded system features, which can 
act as catalysts for a more efficient validation process, can 
be used to complement these developments. 

Government agencies and private industry are currently 
demonstrating automated testing techniques for all phases 
of the flight qualification process. Automated testing 
techniques are being used on the X-31 enhanced fighter 
maneuverability (EFM) program 1 as well as military ad- 
vanced fighter demonstration and aircraft production pro- 
grams. Ames-Dryden’s traditional verification and valida- 
tion techniques 2 have been developed from flight qualifica- 
tion experiences on several experimental flight research ve- 
hicles, including the F-8C digital fly-by- wire aircraft control 
system, the highly maneuverable aircraft technology (Hi- 
MAT) vehicle, the advanced fighter technology integration 
(AFTI) F-16 aircraft, the X-29 aircraft, and the F-18 HARV 
aircraft. This experience provides the cornerstone for de- 
veloping advanced testing techniques. Ames-Dryden rep- 
resents a unique environment for this type of research be- 
cause of the diversity of embedded research systems which 
are flight qualified at the facility. The first digital fly-by-wire 
control system development and test effort used automated 
testing techniques. A software support package integrated 
with the F-8C aircraft simulation introduced automated test- 
ing techniques for the redundancy management logic, 3 Sim- 
ilar techniques were used on the HiMAT program during 
the late 1970’s. 4 Ames-Dryden is currently developing auto- 
mated testing techniques to reduce testing costs and increase 
the availability of an aircraft for flight. Ames-Dryden is 
constructing an integrated test facility (ITF) 5 to develop ad- 
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vanced flight qualification technology emphasizing aircraft - 
in-the-loop techniques. 

Enhancements to the FCS’s testability must accompany 
the developing automated testing techniques. With current 
computing technology, on-aircraft central processing unit 
(CPU) speeds of 20 million instructions/sec are possible us- 
ing very large scale integration (VLSI) based 32-bit micro- 
processors. These speeds will offset the additional software 
overhead typically associated with higher order languages 
and will allow embedded features to improve target system 
and automated testing system integration. 

Nomenclature 


AC 

alternating current 

CPU 

central processing unit 

DC 

direct current 

FCC 

flight control computer 

FCS 

flight control system 

FS/CP 

failure status/control panel 

GUI 

graphical user interface 

HARV 

high alpha research vehicle 

ITF 

integrated test facility 

LED 

light emitting diode 

RFCS 

research flight control system 

RISC 

reduced instruction set computer 

SEU 

system evaluation unit 

SIH 

simulation interface handler 

STIL 

system test interface language 

UMN 

universal memory network 


The X-29 Forward-Swept-Wing Automated 
Testing System 

The X-29 FCS, described in Ref. 6, used off-the-shelf 
hardware. The operational flight program was written in a 
processor-specific assembly language and consisted of 205 
total modules, approximately 29,000 instructions, 220 vari- 
ables, and 3,000 constants. 7 During aircraft ground testing 
for the second X-29 aircraft, automated testing techniques 
reduced the time required for aircraft FCS verification and 
validation tests from 4 weeks to 7 days. This was a labor 
savings of more than 8 man months and allowed the aircraft 
to be flown 3 weeks earlier than would have been possible 
with conventional test techniques, 8 

No modifications were made to the X-29 FCS software 
or hardware to assist in applying the automated testing tech- 
niques. Validation of control systems in a state ready for 
flight is desirable. Frequently performed verification and 
validation tests were the primary focus of automation, in- 
cluding time history, frequency response, and inpul/output 


control system checks. These tests were run for all FCS 
changes. System engineers and control engineers, not au- 
tomated techniques, decided what should be tested. 

A primary goal in developing automated testing tech- 
niques at Ames-Dryden was identifying open-systems tech- 
nology that promotes a generic approach to closed-loop val- 
idation of uniquely configured embedded systems. Sev- 
eral different testing configurations are used for validat- 
ing embedded systems at Ames-Dryden. The primary sim- 
ulation configurations used for testing the X-29 FCS in- 
cluded modeled aircraft dynamics integrated with the actual 
FCS hardware. 

Principal Design Features 

The X-29 automated test system (Fig. 3) was designed 
to help verify and validate the FCS software. It was not 
designed to flight qualify aircraft systems using distributed 
processing techniques such as the F- 1 8 H ARV. The primary 
elements were a nonlinear real-time aircraft simulation with 
data recording capabilities, a Unix^ workstation for simu- 
lation command file generation, the X-29 flight control com- 
puters (FCCs), and a hardware actuator model. The real ac- 
tuators were used during aircraft-in-the-loop testing. 

Minor software modifications to the X-29 aircraft sim- 
ulation were required. When a change was made, it was 
designed to be easily incorporated into other simulations. 
Control of the simulations was automated by allowing the 
simulation executive to automatically read test command 
files instead of a tester manually typing simulation com- 
mands at a keyboard. The similarities in the command-line 
user interfaces inherent in all Ames-Dryden aircraft simu- 
lations made automated test system development for differ- 
ent aircraft possible. Ames-Dryden had adopted a standard 
command line interface to minimize development costs and 
training time when new simulations were required. The au- 
tomated testing system benefited from this standardized user 
interface approach. The simulation commands set internal 
simulation variables. In some cases, commands used cum- 
bersome and nondescriptive array variables internal to the 
simulation. Consequently, the test engineer needed to main- 
tain an extensive working knowledge of the simulation soft- 
ware implementation. 

An advantage of the X-29 automated test system was the 
ability to hide simulation software mechanisms by provid- 
ing a higher order interface. This was accomplished with the 
system test interface language (STIL). Test procedures writ- 
ten in STIL were translated into simulation commands by 
the STIL translator. The STIL translator was written in the 
“C” programming language and was built around the Unix 
utility M4 C language macro preprocessor. The STIL trans- 
lation was executed on the Unix workstation. The output of 
the translator was a file of valid simulation commands that 

® Unix is a registered trademark of AT&T, Bell Laboratories, Murray 
Hills, New Jersey. 
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were transferred lo the simulation computer and read by the 
simulation executive lo run a test. The simulation software 
executive was optimized to increase the speed at which the 
new command files could be read. 

Signal generation code was added to the X-29 aircraft 
simulation program lo produce a variety of signal shapes. 
Frequency sweeps, square waves, biases, doublets, and null 
values could replace or be summed on to the simulation ana- 
log outputs to the FCCs. User-definable attributes such as 
signal amplitudes, pulse widths, pulse durations, and fre- 
quencies could be specified by simulation commands in the 
test command file. Test discrete outputs could be substi- 
tuted for simulation discrete outputs. The flexibility on type 
of signal and when it was introduced to the FCS was useful. 

Test results were interpreted primarily by comparison 
analysis lo baselined data. The actual FCS was compared 
to an independently implemented nonlinear control system 
simulation. Automatically recording the test data results in 
real time, transfering the results to an appropriate analysis 
computer, and generating an ovcrplot of actual and expected 
results were desired. The plotting process w'as optimized lo 
produce several overplots with minimal user intervention. 
FORTRAN graphics routines were the basis for the plotting 
application. Plotting control files which governed the plot- 
ting process were automatically generated and used to drive 
the output plotting device. Other methods of interpreting 
the test results included capturing bit-packed control system 
status words, transfering the data to a Unix workstation, and 
automatically comparing them to expected results. A pass- 
fail message was generated for each test case to allow for 
quick scanning of the test results. 

Development 

The development process for the X-29 automated test sys- 
tem was to design a prototype system using a specific ex- 
ample of a typical verification test. Useful aspects of the 
prototype would then be carried forward to an operational 
phase. In May of 1987, a prototype automated test system 
was demonstrated by Ames-Drydcn. 

The design philosophy for the demonstration system was 
to provide a front end to the X-29 hardware-in-the-loop 
simulation (Fig. 4) with a Unix workstation. A relational 
database management system running on the workstation 
was the primary interface between the user and the auto- 
mated test environment. The workstation provided a menu- 
driven user interface for test generation, control operation, 
results processing, and test documentation archiving. The 
relational database menu items were chosen with several 
combinations of keyboard control sequences. Communica- 
tion between the Unix workstation and the real-time simula- 
tion was a standard RS-232 link. The goal was centralized, 
fully automated control of the entire test process. 

The X-29 demonstration system was designed to auto- 
matically run an open-loop frequency response test across 


the flight control computers (FCCs) and display the results 
plotted against predetermined gain and phase margin lim- 
its. A Unix workstation process called the simulation in- 
terface handler (SIH) provided overall management of the 
test sequencing. Once a STIL test procedure was trans- 
lated into the corresponding simulation commands, the SIH 
would send the command file to the simulation computer. 
The simulation computer would read the test command file 
to run the test, record the data, convert the data to an ASCII 
format, and transfer the data back to the Unix workstation. 
The SIH then initiated the data analysis routines and pre- 
sented the user with a comparison of actual gain and phase 
data overlayed with the predetermined limits. All of this was 
controlled from a single menu-driven user interface, mini- 
mizing user interaction. 

Operational Experiences 

The automated testing technology demonstration for the 
X-29 aircraft highlighted the practical constraints of fully 
automating a process with equipment that was not designed 
for automated control. These practical constraints trans- 
formed the desired fully automated design into a highly in- 
teractive design. Several aspects of the X-29 automated 
test system did not represent a practical solution in an op- 
erational environment. For instance, the RS-232 connec- 
tion between the real-time simulation and the Unix work- 
station was extremely slow for transferring large (1 Mbyte) 
data files. The RS-232 was the only option at the time of 
the demonstration. 

Another constraint was using a relational database for a 
user interface. Selecting menu choices, traversing menu 
pages, and completing the database forms used for writing 
test procedures was accomplished with cumbersome con- 
trol sequences typed on a keyboard. Graphical user inter- 
face (GUI) standards were not readily available. This lack 
of GUI standards prevented GUI development efforts, since 
the results would not be portable to emerging high-speed 
Unix workstations. Use of the database was not carried for- 
ward to an operational phase. 

Practical constraints of the X-29 FCS also became appar- 
ent. These constraints included limitations associated with 
automating pilot switch actions on the failure status/control 
panel (FS/CP). The FS/CP (Fig. 5) was the pilot’s interface 
to the FCCs. This panel was used to reset the digital comput- 
ers, reset or arm the control system actuators, enter discrete 
flap positions, initiate built-in test sequences, and manually 
change control system modes. The interface between the 
FS/CP and the FCCs was a custom designed 1 MHz serial 
bus. Unfortunately, this bus had no provisions for exter- 
nal interfaces. Consequently, typical pilot actions using the 
FS/CP could not be completely automated. Automatically 
resetting and arming actuators and selecting flap positions 
with a thumbwheel were not possible and remained manual 
operations during testing. Since there were several discrete 
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Hap position combinations associated with the X-29 control 
system, lengthy validation tests resulted. 

Minimal hardware changes to the simulation were re- 
quired to allow FCS mode changes to be automated. Chang- 
ing control system modes from primary to backup and re- 
setting the FCCs was successfully automated. The com- 
puter interface console (CIC), which interfaced the FCCs 
to the hardware-in-thc-loop simulation test equipment, per- 
formed signal conversions such as DC to AC, DC to syn- 
chro, and low-voltage to the high-voltage discretes used on 
the aircraft by the FCCs. With simple hardware modifi- 
cations to the CIC, the automated test system could drive 
relays emulating the physical FS/CP switch movement for 
mode changing and FCC resetting. A simulation command 
file could then be used to reset the FCCs. Because the FCCs 
cleared all internal random access memory (RAM) locations 
on startup, automated FCC reinitialization between consec- 
utive test runs was possible. 

The X-29 FCC CPUs were interfaced on the ground with 
a system evaluation unit (SEU) used to debug control system 
software. The FCC memory data and processor register val- 
ues could be monitored in real time on the SEU front panel 
LED display. The data monitoring was limited to a single, 
user-selectable, internal FCC parameter and was not use- 
ful for analyzing relationships between parameters across 
different FCC channels. Hard copy results from the SEU 
interface of internal FCC variables were only available by 
capturing data dumps while the CPUs were stopped. 

The primary method of viewing internal FCC variables 
in real time was a 64-word ARINC 429 bus. FCC soft- 
ware modifications were necessary to change which values 
were loaded on the bus. The 429 bus parameters were cap- 
tured by an extended aircraft interrogation and display sys- 
tem (XAIDS) 9 and relayed to the simulation computer by a 
1553 bus for display on strip charts. 

Controling the simulation with command files was the 
most successful aspect of the X-29 automated testing de- 
velopment. The STIL interface preserved the manual user 
interface already in operation and increased the user’s abil- 
ity to write several similar tests quickly, accomplishing 
validation in a shorter period of time. The STIL, how- 
ever, did not deliver all the capabilities normally associated 
with a programming language and had very limited mathe- 
matical capability with no looping features or other useful 
control constructs. 

The X-29 automated test system acted in an open-loop 
fashion. There were no real-time feedbacks from the test- 
ing environment to provide closed-loop control of the test 
process. Consequently, no error recovery from test sys- 
tem hardware failures, or other erroneous situations, existed. 
The simulation computer executive would not halt the read- 
ing of a test command file once a test began executing but 
always attempted to finish executing a test command file re- 
gardless of the status of the testing environment. 


A good example of the disadvantages of open-loop oper- 
ation occurred during the X-29 aircraft-in-thc-loop testing. 
A command file controlling the simulation commanded a 
change in (light condition. The simulation responded im- 
mediately and proceeded with commands which began ex- 
citing the FCS at the specified (light condition. However, 
manually controlled airdala test equipment connected to the 
aircraft’s pitot-static system had not been adjusted to the de- 
sired (light condition before control system excitation be- 
gan. The simulation computer started executing the test be- 
fore the FCS had internally selected the appropriate gain set. 
Erroneous control system gains resulted in an aircraft limit 
cycle, and manual safety precautions were used to discon- 
tinue the test. This problem could have been avoided if the 
simulation executive had halted test execution until the FCS 
was properly initialized. To do this, feedbacks from the FCS 
confirming that desired (light conditions had been reached 
would be needed. 

The F-18 High Alpha Research Vehicle 
Automated Test System 

Principal Design Features 

The F-18 HARV automated test system is meant to im- 
prove on the X-29 automated testing environment features. 
The development of a test language, closed-loop control 
of the testing environment, graphical user interfaces, and 
quick-look monitoring displays are being emphasized. The 
design will attempt to improve automation of the real-time 
aircraft simulation control and also to add features which au- 
tomate the tester’s decision-making capabilities. The F-18 
HARV automated testing system is being partitioned into 
four general sections: test generation, test engine, test mon- 
itoring, and test data analysis. 

Ames-Dryden is flying the F-18 HARV to perform high- 
angle-of-attack flight research. The basic FCS has been 
extensively modified. A research flight control system 
(RFCS), implemented with Ada® and interfaced to the ba- 
sic FCS, will be used for aircraft thrust vectoring control. 
The F-18 HARV automated test system is currently being 
developed to help verify and validate both F-18 HARV con- 
trol systems. 

Open systems architectures and software standards are 
being followed when possible to insure portability to other 
computing platforms. The test concept for the F-18 HARV 
continues the integrated simulation and automated testing 
environment approach. An overview of the F-18 HARV 
hardwarc-in-the-Ioop simulation can be seen in Fig. 6. The 
X-29 automated test system did not address validation of 
concurrent processes such as those found in the F- 18 HARV 
with the mission computer, FCC, and research FCC. The 


® Ada is a registered trademark of the United SLates Government Depart- 
ment of Defense. 
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F-18 HARV testing methodologies will address the trends 
toward distributed processing vehicle management systems. 

Initial Development 

An interativc development approach is being taken to the 
automated testing system. As useful features are designed 
and implemented, they arc integrated with the F-18 HARV 
testing environment in a buildup fashion. This will insure 
usability and, more importantly, acceptance by the test en- 
gineers in an operational environment. 

The integrated F-18 HARV automated test system and 
simulation (Fig. 7) consists of several computers inter- 
faced to a high-bandwidth reflective memory network. 
Mainframe computers provide the functions of real-time 
input/output to the FCCs, data recording of all testing envi- 
ronment parameters, and the nonlinear aircraft simulation. 
The Unix RISC-based workstation provides test environ- 
ment control and monitoring. 

The reflective memory network or the universal memory 
network (UMN) connects computers such as mainframes 
and Unix workstations. It is a reflective memory system 
with composite rates of up to 40 Mbytes/sec. Processors 
connected to the UMN can effectively share a global mem- 
ory partition with no special protocols or additional proces- 
sor overhead. The UMN is currently operational in the F-18 
HARV real-time simulation and is being used in conjunction 
with a high-speed data recording capability also developed 
forthelTF. The F-18 HARV automated testing environment 
will be designed around this high-speed memory network to 
overcome the data transfer, lest monitoring, and lest control 
deficiencies of the X-29 automated test systems. 

Test generation is focused on improving the X-29 STIL 
concept of developing efficient methods of writing test pro- 
cedures. The ability of the X-29 simulation executive to 
read command files was duplicated in the F-18 HARV sim- 
ulation executive. Currently, a longer term solution to pro- 
viding testers with a test language is being developed. The 
test language will support common higher order program- 
ming languages such as FORTRAN and C. For efficient 
use of the F-18 HARV testing environment, user libraries 
will provide the test engineer with access to automated test- 
ing features. These libraries, callable from common higher 
order languages, will hide the complexities of controlling 
the automated environment and will allow users to write 
test procedures to precisely control the validation process. 
This approach minimizes test language development time 
while providing a full set of programming control features. 
The test engine will compile the test procedure and produce 
the commands necessary to coordinate and control the auto- 
mated test environment. 

The lest engine is a continuation of the X-29 simulation 
interface handler concept. The test engine will control real- 
time data recording, aircraft simulation, test monitoring, and 
any necessary real-time test data analysis processes. It is 


a high speed Unix workstation with a library of test func- 
tions used to obtain closed-loop control of the automated 
test environment. The test environment feedbacks will in- 
clude real-time simulation values, FCS values, and opera- 
tional status parameters from the automated testing environ- 
ment. The test engine will also be used to provide the user 
interface. Figures 8(a) and 8(b) show the contrast between 
the manually controlled simulation and the closed-loop au- 
tomated simulation control. The test engine hardware will 
be a RISC-based filcserver connected to a RISC-based Unix 
workstation providing the user interface. The lest engine 
will be interfaced to the testing environment with the UMN. 

The test monitoring and analysis functions will be driven 
by real-time current value tables (CVTs) located in the UMN 
reflected global memory partition. The design goal is to 
improve the quick-look capability at test data results and 
to provide several types of user-customizable displays in 
real time. This will provide the user with feedback to 
avoid rerunning lengthy tests because of set up errors or bad 
data. Various types of monitoring applications will be de- 
veloped using commercial off-the-shelf applications. The 
F-18 HARV 1553 buses will be interfaced to the UMN to 
provide real-time information from the onboard FCSs and 
avionics. Because of the increases in Unix workstation com- 
puting performance over the last three years, implementing 
GUI standards is now feasible. The X-window-based appli- 
cations are being chosen to provide a flexible multiwindow 
user interface. 

The F-18 HARV test system is improving on the X-29 
automated test system. Closed-loop control of the testing 
process will provide error recovery capability and give the 
necessary decision control to the test engine to better allow 
automation of failure modes and effects tests. The point 
and time when failures can be introduced can be controlled 
based on aircraft conditions. The automated lest system sig- 
nal generation modifications incorporated in the X-29 air- 
craft simulation program were designed to be easily trans- 
fered to the F-18 HARV simulation. While new F-18 HARV 
automated test system advancements are being developed, 
command file control of this signal generation code can be 
used in parallel with new developments. 

Flight Control System Design 
Recommendations 

Experiences with the X-29 automated testing capabili- 
ties have shown that more elegant approaches of combining 
the embedded system design and test requirements at earlier 
stages of development are needed. A major key to testabil- 
ity is participation of test personnel in the design process. 
If validation tools and techniques are identified during the 
initial stages of embedded system implementation, valida- 
tion can be made easier. The target systems and the test 
systems must be considered one development effort. Well- 
structured top-down embedded system design with modu- 
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larity increases the ability to maintain the software. But val- 
idation is not made easier unless testability was considered 
during the design and coding stages. Higher order languages 
like Ada arc an attempt to reduce software life cycle costs 
by increasing the readability and understandability of em- 
bedded code, but testability must also be addressed, 

Nonreal-Time Considerations 

Most features that would significantly increase testabil- 
ity are inexpensive to incorporate, but would require more 
discipline from the software engineering perspective. When 
validating a complex software system, online access to the 
information describing system implementation details and 
expected operations is needed. Often, design documen- 
tation is incomplete, easy to misunderstand, and difficult 
to piece together in a coherent fashion. Consistent, well- 
structured internal documentation of functional elements 
would allow validation tools to perform searches quickly 
to answer simple questions about how the embedded sys- 
tem should operate. For example, software off-the-shelf 
component data books should be established to complement 
the now emerging object-oriented programming (OOP) ap- 
proaches. Data dictionaries are vital in the management of 
embedded system information. These dictionaries should 
describe all internal program variables, scale factors, max- 
imum and minimum values, addresses, set and used infor- 
mation, bit packing descriptions, and a contextual comment 
on how the variable is used. Automated ways of updating 
the dictionary should be linked to program generation. A 
well-structured format allowing validation tools to parse the 
dictionary is required. 

Real-Time Considerations 

Validation normally adopts black box testing techniques. 
However, some validation tests require insight to the com- 
puting systems. The trend to segregate the redundancy man- 
agement and mode logic techniques from the control law ap- 
plication software is continuing. System partitioning of this 
nature was proven valid with the X-29 FCS. The X-29 FCS 
used separate processors for input/output and control law 
execution. The F-18 HARV project will also demonstrate 
the validity and benefits of this type of embedded system 
partitioning with the RFCS. Redundancy management and 
mode logic functions account for a large percentage of val- 
idation test cases. These functions rarely change during the 
course of flight testing. Test cases for the redundancy man- 
agement of embedded systems are typically generated using 
insight to the implementation of the redundancy manage- 
ment. The test case requirements are strongly influenced by 
knowledge of the internal software logic. To automatically 
test the input/output logic of an embedded system, monitor- 
ing and independently controlling all of the input/output sig- 
nals in real time is vital. Control laws are traditionally tested 
with the black box approach. Time history and frequency 
response test case requirements are influenced more by the 


aircraft’s envelope and operation than by internal software 
logic. The automatic testing of control laws is easier to 
achieve. 

Real-time unobtrusive access to internal variables was 
needed several times during the X-29 validation and flight 
lest process. In several cases, internal intermediate variables 
needed to be examined. For instance, the accuracy of an on- 
board analytical actuator monitor had to be verified during 
closed-loop dynamic maneuvers. This verification required 
software modifications to instrument the code. During flight 
test, surface command reasonability checks were tripping 
during the take-off roll, causing a down-mode to the analog 
backup control system. Analysis of how close the moni- 
tor was to tripping was needed as the aircraft taxied. The 
use of a temporary storage variable for multiple interme- 
diate calculations should be avoided to facilitate real-time 
external monitoring. Tradeoffs in memory and timing con- 
straints may be less critical with the advent of higher density 
memories and faster processors. 

In most cases, modifications to the X-29 real-time soft- 
ware were made to output the required variables on an 
ARINC 429 bus. Sixty-four 16-bit words could be out- 
put. Four modules, each executing at 40 Hz, were used to 
load FCC output buffer registers for use by downlink instru- 
mentation. In the X-29 validation lab, the ARINC 429 bus 
signals were relayed to the real-time simulation computer 
through a 1553 bus for display on strip charts. The primary 
use of the ARINC bus was to downlink vital signs of the 
FCS for in-flight monitoring. Methods allowing different 
429 bus variables to be selected without requiring software 
modifications were considered, but never implemented. 

As described in Ref. 10, high-performance experimen- 
tal aircraft programs have traditionally relied on parame- 
ter estimation techniques to determine aircraft stability for 
safety of flight envelope expansion. During the X-29 enve- 
lope expansion, intermediate control system variables on the 
ARINC 429 bus were downlinked to perform near-real-time 
longitudinal frequency response measurements to assess ve- 
hicle stability characteristics. This capability required con- 
trol system code instrumentation to capture the correct val- 
ues for monitoring. Well-positioned, selectable software 
monitoring points would have avoided the need for control 
system changes and would have allowed for other uses of 
this type. The F-18 HARV FCS has a programmable fea- 
ture by which 64 different variables can be requested from 
the basic FCS by the RFCS for output on the aircraft’s 1553 
avionics bus. However, to change the set of variables a re- 
compilation of the RFCS software is necessary. Recompi- 
lation is not desirable for software under test. 

Lengthy post-test data dumps are difficult to analyze. Af- 
ter a test is run, events may not be accurately remembered. 
Real-time data analysis offers more flexibility and would in- 
crease productivity in tracing real-time execution when ap- 
plying troubleshooting techniques. Vital signs of the soft- 
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ware should be monitored in real Lime. Dynamically choos- 
ing real-time monitoring points and transporting the data to 
an engineering workstation is desirable. 

Flight Control and Test System Synchronization 

The aircraft simulation and flight control computers had 
to be synchronized for the X-29 hardwarc-in-thc-loop con- 
figuration. Automated open-loop frequency response tests 
showed erroneous phase margins at higher frequencies be- 
cause of unpredictable timing relationships between the 
simulation and FCCs. Fortunately* an 80-Hz synchroniza- 
tion output discrete was generated by the FCS and was used 
to drive the simulation real-time executive. This allowed for 
more predictable timing relationships and helped to correct 
the problem. 

Timing relationships between the test system and embed- 
ded system can be critical. An automated test system must 
have the ability to introduce an error function at any point in 
the real-time cycle of the FCS software. During hardware- 
in-the-loop testing of the X-29, a high-frequency pulse to 
a canard position feedback was discovered to cause loss of 
control. This failure scenario only occurred 50 percent of 
the time. Loss of control was dependant on when the failed 
surface position input was sampled by the flight control sys- 
tem. Complex software modifications were required to use 
spare discrete and analog input signals for fault introduc- 
tion. To completely automate failure modes and effect test- 
ing, these situations can be avoided if the embedded system 
and target system are interfaced correctly. 

Embedded schemes, allowing spare input and output sig- 
nals to be used by the automated test system, are desirable. 
For example, techniques are being developed to reserve ex- 
ternally controlled input and output discretes to automat- 
ically force miscomparisons of bit-for-bit voting planes. 
Other validation requirements are concerned with timing 
analysis. Appropriate hardware or software interfaces to al- 
low analysis of standard case and worst case execution times 
arc typically exercised during verification. System synchro- 
nization issues require visibility into the internal operation 
of the embedded system. The X-29 SEU could generate out- 
put discretes based on CPU program counter information to 
drive digital timers. To automatically address timing issues, 
appropriate hardware and software interfaces such as this 
are needed. 

Software Instrumentation 

The real-time characteristics of the onboard software in- 
hibit reliably applying automatic code instrumentation after 
the software has been constructed. Automatic code instru- 
mentors would need to know all subtle timing relationships. 
Code instrumentation should be done in conjunction with 
system design to assure these liming relationships are not 
disturbed. As a flight release goes through several changes 
in the operational phase of a program, software test points 
which have initially been added to the code may no longer 


be in the correct place to automatically test a change. To 
eliminate or reduce these adverse effects software test points 
should be included during the software development. 

Concluding Remarks 

Automated closed-loop validation of man-rated flight 
control systems is being done at the NASA Ames Research 
Center Dryden Flight Research Facility. Operational ex- 
periences in developing and using these automated testing 
techniques have highlighted the need for incorporating tar- 
get system features to improve testability. Improved target 
system testability can be accomplished with the addition of 
nonreal-time and real-time features. Online access to target 
system implementation details, unobtrusive real-time access 
to internal user- selectable variables, and proper software in- 
strumentation are all desirable features of the target system. 
Also, test system and target system design issues must be ad- 
dressed during the early stages of the target system develop- 
ment. Processing speeds of up to 20 million instructions/sec 
and the development of high-bandwidth reflective memory 
systems have improved the ability to integrate the target sys- 
tem and test system for the application of automated testing 
techniques. New methods of designing testability into the 
target systems are required. 
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Fig. 1 The X-29 forward-swept-wing aircraft. 



Fig. 2 The F- 18 HARV aircraft. 
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Fig. 4 The X-29 hardware-in-lhc-loop simulation. 
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Fig. 6 Overview of the F-18 HARV hardware-in-the-loop simulation. 
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Fig. 7 The F-18 HARV automated test system functional diagram. 
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